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Test  and  Evaluation: 

Transforming  to  Enable  Successful 
Systems  Development  and  Fielding 

MG  Roger  A.  Nadeau 

Commanding  General,  U.S.  Army  Test  and  Evaluation  Command, 

Alexandria,  Virginia 

The  key  to  making  transformational  changes  in  the  test  and  evaluation  process  is  one 
program  at  a  time.  Army  Test  and  Evaluation  Command  (ATEC)  is  looking  at  itself  to 
further  applications  of  the  principles  of  testing  and  evaluation  discussed  in  this  article. 

Personnel  of  the  Command  are  being  further  trained  in  these  principles  to  facilitate  a  closer 
working  relationship  with  industry,  government  agencies,  and  Project  Program  Managers. 


Army  Transformation  is  not  just  two 
words  on  a  briefing  chart  —  it  is  a 
reality.  The  U.S.  Army  has  been  in  the 
act  of  a  major  transformation  for 
several  years.  Every  organization  in 
the  Army  is  changing  its  processes  and  procedures  to 
facilitate  the  transformation  process.  The  Army  Test 
and  Evaluation  Command  (ATEC)  is  no  exception. 

Although  ATEC  has  implemented  several  transforma¬ 
tional  changes  in  support  of  an  army  at  war,  there  is  much 
more  that  can  be  done.  It  is  not  change  for  the  sake  of 
change.  Every  change  should,  to  some  degree,  save  time  and/ 
or  money,  without  sacrificing  the  quality  of  our  mission. 

ATEC’s  mission  is  simple  —  assess  the  performance 
capabilities  of  every  piece  of  equipment  used  in  any 
way  by  Soldiers.  The  two  basic  questions  we  ask  are: 
Does  it  work?  How  do  I  know? 

Our  end-state  assessments  address  effectiveness, 
suitability,  and  survivability.  Based  on  that  assessment, 
Army  leadership  decides  if  Soldiers  will  ultimately  use 
the  item. 

Throughout  the  process  of  doing  our  job,  we  work 
closely  with  industry,  requirements  developers,  Pro¬ 
gram  Executive  Officers,  rapid  equipping  organiza¬ 
tions,  and  others  who  are  involved  in  determining  the 
need  for  or  developing  materiel  solutions  to  provide 
capabilities  to  the  Army  and  other  Services.  Although 
usually  productive,  the  relationship  between  ATEC 
and  the  aforementioned  groups  can  improve. 

Changing  the  perception 

The  major  challenge  right  now  in  improving  our 
relationship  with  others  is  one  of  perception.  We 


believe  most  agencies  see  ATEC  as  a  hindrance,  rather 
than  an  enabler,  in  providing  equipment  to  satisfy  Soldier 
needs.  Some  of  the  perception  challenges  are  rooted  in 
current  policies  and  regulations.  As  a  general  mle,  most 
mandated  test  events,  developmental  and  operational,  are 
scheduled  just  before  a  major  milestone  decision;  and  no 
independent  agency  test  events  are  required  before 
Milestone  A  at  all.  Industry  and  government  materiel 
developers  must  work  through  technology  and  design 
growth  challenges  early  in  the  program,  often  challenged 
by  schedule  compression  and  funding  constraints,  leaving 
no  schedule  or  funding  flexibility  by  the  time  a  mandatory 
test  event  occurs.  The  time  and  cost  due  to  the  test  and 
evaluation  (T&E),  and  performance  shortfalls  revealed  in 
test,  invariably  creates  the  perception  that  the  test  agency 
is  an  impediment  to  progress.  The  challenge  —  change 
the  perception  by  involving  the  T&E  throughout  the 
acquisition  lifecycle  as  a  proactive  part  of  the  process. 

ATEC  is  aggressively  reaching  out  to  industry  and 
government  agencies,  where  appropriate,  to  prove  the  value 
of  bringing  the  test  community  in  early  in  the  requirements 
development,  equipment  design,  and  development  process¬ 
es.  If  brought  into  the  process  early  and  used  throughout 
the  acquisition  lifecycle  as  a  partner,  we  are  convinced  we 
can  aid  in  building  quality  into  the  product  from  the 
beginning,  saving  time  and  money  and  minimizing  costly 
redesign  when  major  performance  problems  go  undetected 
until  late  in  the  developmental  cycle. 

Early  and  constant  involvement  in  the 
acquisition  process 

As  we  work  to  earn  (and  we  mean  earn)  the  right  to 
be  brought  into  the  acquisition  process  earlier,  ATEC 
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is  also  revisiting  how  we  test.  The  current  process  is 
generally  linear.  Developmental  Testing  (DT)  is 
separate  from,  and  always  prior  to,  Operational 
Testing  (OT).  The  evaluation  process  in  many  cases 
begins  after  testing  is  complete.  Industry  or  other 
government  agency  test  data  are  generally  not 
considered.  Early  coordination  between  industry, 
government  materiel  developers,  and  ATEC  concern¬ 
ing  T&E  methods  and  expectations  often  fails  to 
occur.  These  traditional  processes  are  inefficient, 
archaic,  not  transformational,  and  are  no  longer 
acceptable.  They  waste  time  and  money  and  certainly 
do  not  foster  team  spirit. 

Change  needs  to  start  from  the  beginning.  ATEC’s 
involvement  should  start  in  developmental  activities 
prior  to  Milestone  A.  How  much  involvement  will  vary 
according  to  the  nature  of  the 
end  item.  From  that  point, 

ATEC’s  involvement  should 
be  constant  through  every  mile¬ 
stone  and  all  aspects  of  the 
phases  between  the  milestones. 

Generally  speaking,  we  be¬ 
lieve  Project/Product  Managers 
should  always  have  to  deliber¬ 
ately  decide  not  to  have  an 
ATEC  representative  in  meet¬ 
ings  vice  the  alternative  of  always  having  to  remember 
to  invite  an  ATEC  representative  to  a  meeting. 

The  same  approach  applies  to  ATEC  involvement 
with  the  defense  industry.  The  government  can  include 
lessons  learned  from  T&E  of  similar  equipment  in  the 
advertisement  of  Requests  For  Proposals  to  aid 
industry  proposal  responses.  Once  under  contract, 
dialogue  with  ATEC  can  help  guide  the  thought 
process  in  contractor  testing  which,  in  turn,  can 
influence  progressive  design  changes  to  enhance  system 
performance.  Early,  coordinated  government/industry 
test  planning  can  ensure  common  test  procedures, 
which  will  allow  for  potential  use  of  contractor  data  in 
government  test  performance  reporting,  minimize 
repetitive  testing,  and  better  indicate  system  readiness 
for  government  testing.  ATEC  involvement  in  con¬ 
tractor  testing  also  allows  for  early  development  of  data 
collection  and  modeling  and  simulation  tools  to 
support  informed  development  and  acquisition  deci¬ 
sions  throughout  the  life  of  the  program.  Once  an  item 
enters  government  DT/OT,  appropriate  timely  feed¬ 
back  to  the  prime  contractor  can  be  helpful  in 
identifying  corrective  actions  to  fix  emerging  problems, 
saving  everyone  time  and  money. 


Internal  transformation 

Processes  internal  to  ATEC  need  to  change  as  well. 
Consistent  with  the  December  22,  2007  Test  and 
Evaluation  Policy  Revisions  memo  jointly  signed  by 
the  USD  (AT&L)  and  the  DOT&E,  how  we  do  what 
we  do  must  be  less  linear  and  less  parochial  than  in 
times  past.  Core  T&E  principles  spelled  out  in  the 
OSD  policy  memo  include: 

1.  DT/OT  integration  throughout  program  lifecycle 

2.  Compare  to  current  capabilities 

3.  Focus  -  Measure  improvements  to  capability 

4.  Evaluate  in  mission  context  at  time  of  fielding 

5.  Use  all  available  information 

6.  Exploit  the  benefits  of  M&S 

Transformational  change  internal  to  ATEC  is 

mandatory  to  keep  pace  with  the  transforming  Army. 

Anything  less  is  a  disservice 
to  the  Soldiers  we  ultimately 
serve.  In  the  end,  we  are 
convinced  that  the  quality  of 
our  services  will  increase, 
saving  time  and  money  too. 
Is  this  easy?  No. 

Is  it  the  right  thing  to  do? 
Yes. 

Will  it  take  time?  Yes. 

The  Army  Test  and  Eval¬ 
uation  Command  is  committed  to  this  transformational 
challenge.  The  process  has  begun  and  is  picking  up  steam 
every  day!  □ 
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Success  for  ATEC,  and  the  Army,  is 
being  recognized  as  a  trusted  T&E 
advisor  to  the  acquisition/develop- 
mental  team  to  assist  in  program  cost, 
schedule,  and  performance  risk  man¬ 
agement. 
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